Stereoscopic imaging method and system that divides a pixel matrix into subgroups

ABSTRACT

A stereoscopic imaging method where a pixel matrix is divided into groups such that parallax information is received by one pixel group and original information is received by another pixel group. The parallax information may, specifically, be based on polarized information received by subgroups of the one pixel, group and by processing all of the information received multiple images are rendered by the method.

TECHNICAL FIELD

The present invention relates to an imaging method, and morespecifically, to an imaging method for imaging a subject as astereoscopic image.

BACKGROUND ART

In the related art, a system is proposed in which two video cameras thatare arranged left and right simultaneously image a common subject andthe obtained two kinds of images (a right-eye image and a left-eyeimage) are output to be displayed as a stereoscopic image. In addition,a stereoscopic capturing device is suggested in which an optical systemis shared by combining polarization filters that perform polarization soas to be placed in an orthogonal relationship to each other in order toeasily adjust a lens system for performing stereoscopic capturing (forexample, refer to JP-B-6-054991).

In addition, a method is proposed which aims to perform stereoscopiccapturing with an imaging device composed of two lenses and one imagingmeans (for example, refer to JP-A-2004-309868). The imaging devicedisclosed in the Japanese Examined Patent Application Publicationincludes imaging means having pixels arranged on an imaging plane to thenumber corresponding to integer times of a predetermined number ofscanning lines; first horizontal component polarizing means adapted totransmit only a horizontal component of light of a first optical imagefrom a subject; and first vertical component polarizing means arrangedat a position separated from the first horizontal component polarizingmeans by a predetermined distance and adapted to transmit only avertical component of light of a second optical image from the subject,wherein the horizontal component transmitted by the first horizontalcomponent polarizing means is converged to the pixels in a predeterminedarea on the imaging plane; and the vertical component transmitted by thefirst vertical component polarizing means is converged to the pixels inthe remaining area excluded from the predetermined area. Specifically, ahorizontal component polarizing filter and a vertical componentpolarizing filter arranged separate as far as an interval according tothe parallax of a person are provided with two lenses at a positionapart from a predetermined distance from an imaging plane of a CCD.

CITATION LIST Patent Literature

[PTL 1] JP-B-6-054991

[PTL 2] JP-A-2004-309868

SUMMARY OF INVENTION Technical Problem

However, in the technology disclosed in JP-B-6-054991, the lens systemis shared by overlapping the outputs of two polarization filters andthen combining the optical paths thereof. However, it is necessary toprovide another polarization filter in the lower part to extract aright-eye image and a left-eye image and to make light incident to eachpolarization filter by dividing the optical paths again. In thetechnology disclosed in JP-A-2004-309868, two pairs of the combinationof a lens and a polarization filter are necessary. For this reason, insuch an imaging device, the optical axes of the optical paths, focallength, transmittance, F-number, zoom, diaphragms, focus, convergenceangle, and the like of the two pairs have to completely correspond toeach other, and it is difficult to suppress the occurrence of visualfield competition. Herein, visual field competition refers to aphenomenon in which, for example, when a subject such as a watersurface, a window, or the like that reflects P-wave components butabsorbs S-wave components is imaged, and when an image obtained fromP-wave components and an image obtained from S-wave components arepresent to both eyes, fusion of the images does not occur in the casewhere the luminance thereof is remarkably different, images are shownalternately due to the fact that only one image is superior, or imagessuppress each other in overlapping areas. In addition, since severalpolarization filters are used, there is a problem in that the amount oflight that reaches the imaging means (imaging element) drasticallydecreases.

Therefore, the objective of the present invention is to provide animaging method that can suppress the occurrence of visual fieldcompetition, and prevent drastic reduction in the amount of light thatreaches the imaging elements.

Solution to Problem

Thus, an object is to provide a parallax imaging method, comprisingreceiving a parallax information by a first pixel group of a pixelmatrix; receiving an original information by a second pixel group of thepixel matrix, wherein the parallax information is calculated based on afirst polarized information that is received by a first pixel subgroupof the first pixel group and a second polarized information that isreceived by a second pixel subgroup of the first pixel group; andprocessing the original information with the first parallax informationand the second parallax information to respectively render a first imageand a second image. The first pixel group of the pixel matrix mayinclude at least a pixel row, and the second pixel group of the pixelmatrix may comprise of pixel rows not included in the first pixel group.

Further, the first pixel group of the pixel matrix may include at leasta pixel row for every N-th row, where

-   N≧2,

and the second pixel group of the pixel matrix may comprise pixel rowsnot equal to every N-th row. An upper limit of N may be

-   N=2^n,

where n is a natural number from 1 to 5 and, specifically n may equal 3.

Furthermore, a direction of an electronic field of the first polarizedinformation may be orthogonal to a direction of an electronic field ofthe second polarized information.

Another object is to provide a parallax image apparatus that maycomprise a set of pixels disposed in a matrix, a first image pixel groupof the set of pixels for receiving original information; and a secondimage pixel group of the set of pixels for receiving a parallaxinformation, wherein the original information received by the firstimage pixel group is converged light that passes a first polarizationmeans and a second polarization means and the first image pixel groupconverts the converged light to electrical signals, wherein the parallaxinformation received by the second image pixel group is converged lightthat passes a first polarization means and the second image pixel groupconverts the converged light to electrical signals, and a processor forprocessing the original information with the parallax information torender a first image and a second image. The apparatus may be one of adigital camera, a personal computer, a mobile terminal equipment, avideo camera, or a game machine.

Another object is to provide a parallax imaging system that may comprisea set of pixels disposed in a matrix, a first image pixel group of theset of pixels for receiving original information; and a second imagepixel group of the set of pixels for receiving a parallax information,wherein the original information received by the first image pixel groupis converged light that passes a first polarization means and a secondpolarization means and the first image pixel group converts theconverged light to electrical signals, wherein the parallax informationreceived by the second image pixel group is converged light that passesa first polarization means and the second image pixel group converts theconverged light to electrical signals, wherein the first polarizationmeans has a first area and a second area arranged along a firstdirection, and wherein the second polarization means has a third areaand a forth area arranged along a second direction.

Another object is to provide a parallax imaging apparatus that maycomprise a first image pixel group for receiving original information;and a second image pixel group for receiving a parallax information,wherein the original information received by the first image pixel groupis converged light that passes a first polarization means and a secondpolarization means and the first image pixel group converts theconverged light to electrical signals, wherein the parallax informationreceived by the second image pixel group is converged light that passesa first polarization means and the second image pixel group converts theconverged light to electrical signals, wherein the first polarizationmeans has a first area and a second area arranged along a firstdirection, and wherein the second polarization means has a third areaand a forth area arranged along a second direction.

Another object is to provide a non-transitory computer readable mediumstoring program code that when executed by a computer performs anparallax imaging process in a parallax system comprising a set of pixelsdisposed in a matrix, wherein the set of pixels has a first pixel groupand a second pixel group, where the process may comprise receiving aparallax information by the first pixel group of a pixel matrix;receiving an original information by the second pixel group of the pixelmatrix, wherein the parallax information is calculated based on a firstpolarized information that is received by a first pixel subgroup of thefirst pixel group and a second polarized information that is received bya second pixel subgroup of the first pixel group; and processing theoriginal information with the first parallax information and the secondparallax information to respectively render a first and second image.

BRIEF DESCRIPTION OF DRAWINGS

(A), (B), and (C) of FIG. 1 are respectively a conceptual diagram of animaging device of Embodiment 1 and diagrams schematically showingpolarization states in a first polarization means and a secondpolarization means.

(A) and (B) of FIG. 2 are respectively a conceptual diagram of lightthat passes a first area in the first polarization means and a thirdarea in the second polarization means and reaches an imaging elementarray, and a conceptual diagram of light that passes a second area inthe first polarization means and a fourth area in the secondpolarization means and reaches the imaging element array in the imagingdevice of Embodiment 1, and (C) and (D) of FIG. 2 are diagramsschematically showing images formed in the imaging element array bylight shown in (A) and (B) of FIG. 2.

(A) and (B) of FIG. 3 are respectively a schematic partialcross-sectional diagram of an imaging element and a diagramschematically showing an arrangement state of wire-grid polarizersaccording to the imaging device of Embodiment 1.

FIG. 4 is a conceptual diagram of an imaging element array with a Bayerarrangement in the imaging device of Embodiment 1.

FIG. 5 is a conceptual diagram of the imaging element array with a Bayerarrangement to perform a demosaicing process for electronic signalsobtained from imaging elements and to describe an image process forobtaining signal values.

(A) and (B) of FIG. 6 are respectively diagrams each schematicallyshowing polarization states in a first polarization means and a secondpolarization means provided in an imaging device of Embodiment 2.

FIG. 7 is a conceptual diagram of an imaging element array with a Bayerarrangement in the imaging device of Embodiment 2.

(A) to (D) of FIG. 8 are schematic diagrams of a first polarizationmeans provided in an imaging device of Embodiment 3.

(A) and (B) of FIG. 9 are diagrams substituting photographs of left-eyeimages and right-eye images showing a result of the relationship betweenan extinction ratio and parallax in Embodiment 4.

(A), (B), and (C) of FIG. 10 are graphs each showing results of therelationship between a pitch of wires composing a wire-grid polarizer, awavelength of incident light, and an extinction ratio, the relationshipbetween the height of the wires composing the wire-grid polarizer, thewavelength of incident light, and the extinction ratio, and therelationship between the (width/pitch) of the wires composing thewire-grid polarizer, the wavelength of incident light, and theextinction ratio in Embodiment 5.

FIG. 11 is a graph showing the result of the relationship between thelength of two wires composing the wire-grid polarizer, the wavelength ofincident light, and the extinction ratio in Embodiment 5.

FIG. 12 is a conceptual diagram of an image element array with a Bayerarrangement in an imaging device of Embodiment 6.

FIG. 13 is a conceptual diagram of an image element array with a Bayerarrangement in a modified example of the imaging device of Embodiment 6.

(A) and (B) of FIG. 14 are respectively schematic partialcross-sectional diagrams of an image element in the modified example.

DESCRIPTION OF EMBODIMENTS

Hereinbelow, the present invention will be described based on theEmbodiments with reference to the drawings, but the invention is notlimited to the Embodiments, and the various numeric values and materialsin the Embodiments are examples. Furthermore, description will beprovided in the following order.

1. Imaging Method of the Present Invention and General Description

2. Embodiment 1 (Imaging method of the present invention)

3. Embodiment 2 (Modification of Embodiment 1)

4. Embodiment 3 (Another modification of Embodiment 1)

5. Embodiment 4 (Another modification of Embodiment 1)

6. Embodiment 5 (Another modification of Embodiment 1)

7. Embodiment 6 (Another modification of Embodiment 1), and Others

IMAGING METHOD OF THE PRESENT INVENTION, AND GENERAL DESCRIPTION

The value of N is not limited in the imaging method of the invention,but the value is set to an integer equal to or greater than 2, and theupper limit can be 2⁵. Alternatively, the value of N is not limited inthe imaging method of the invention, but N=2^(n), and n can be a naturalnumber from 1 to 5.

In the imaging method of the invention with the preferable configurationdescribed above, an optical system includes:

(a) a first polarization means which polarizes light from a subject,and;

(b) a lens system which converges light from the first polarizationmeans,

in which a first image element group has a second polarization means inthe side of the light incidence, and converts light converged by thelens system to electrical signals (to be more specific, the first imageelement group converts light that is converged by the lens system andpasses the first polarization means and the second polarization means toelectrical signals), a second image element group converts light that isconverged by the lens system to electrical signals (to be more specific,the second image element group converts light that is converged by thelens system and passes the first polarization means to electricalsignals), the first polarization means has a first area and a secondarea arranged along a first direction, a polarization state of a firstarea passing light that passes the first area and a polarization stateof a second area passing light that passes the second area aredifferent, the second polarization means has a plurality of third andfourth areas extending in the first direction, a polarization state of athird area passing light that passes the third area and a polarizationstate of a fourth area passing light that passes the fourth area aredifferent, the first area passing light passes the third area andreaches the first image element group, the second area passing lightpasses the fourth area and reaches the first image element group, andaccordingly, parallax information can be obtained in which a distancebetween the barycenter of the first area and the barycenter of thesecond area is set to a base-line length of parallax of both eyes.Furthermore, an imaging device including an optical system with theabove embodiment may be called “an imaging device in the presentinvention” for the sake of convenience.

According to the imaging device in the invention, since the imagingdevice is composed of one pair of the first polarization means and thesecond polarization means, and one lens system, it is possible toprovide an imaging device which is monocular, small, and has a simpleconfiguration and structure. In addition, since two pairs of thecombination of a lens and a polarization filter are not necessary, nodeviations or differences occur in zoom, diaphragms, focus, convergenceangle, or the like. Moreover, since the base-line length of the parallaxof both eyes is relatively short, a natural stereoscopic effect can beobtained. Furthermore, two-dimensional images or three-dimensionalimages can be easily obtained by attaching or detaching the firstpolarization means.

Herein, according to the imaging device of the invention, it ispreferable to employ an embodiment that the first polarization means isarranged around the diaphragm of the lens system. Alternatively, whenlight incident to the lens system is once assumed to be parallel light,and finally converged (forms image) on an imaging element, it ispreferable to arrange the first polarization means in the lens systemportion being in the state of parallel light. In such an embodiment,generally, it is not necessary to re-design the optical system of thelens system, and a change can be performed in mechanical (physical)design by fixing the first polarization means to the existing lenssystem or attaching the first polarization means detachably.Furthermore, in order to attach the first polarization means on the lenssystem detachably, for example, the first polarization means may beconfigured or structured to be similar to diaphragm blades and arrangedwithin the lens system. Alternatively, the lens system can be configuredor structured such that a member provided with the first polarizationmeans and an opening together is attached to a rotary axis so that themember can rotate around the rotary axis parallel with the optical axisof the lens system, and a light beam passing through the lens systempasses the opening by rotating the member around the rotary axis, orpasses the first polarization means. Alternatively, the lens system canbe configured or structured such that the member provided with the firstpolarization means and the opening together is attached slidably to thelens system in a direction orthogonal to, for example, the optical axisof the lens system, and a light beam passing through the lens systempasses the opening or passes the first polarization means by sliding themember.

According to the imaging device of the invention with the preferableembodiment described above, in the first polarization means, a centerarea is provided between the first area and the second area, and apolarization state of a center area passing light that passes the centerarea can be configured not to change from a state before being incidentin the center area. In other words, the center area can be in afree-passing state of polarization. In the center area of the firstpolarization means, light intensity is strong, but a parallax amount issmall. Therefore, by employing such an embodiment, it is possible toincrease the light intensity that the image element array receives, andto secure a sufficient base-line length of the parallax of both eyes.When the external shape of the first polarization means is circular, thecenter area can be shaped to be circular, the first area and the secondarea can be a fan shape of which the central angle surrounding thecenter area is 180 degrees, the center area can be a square or a rhombusshape, and the first area and the second area can be shaped similar to afan shape of which the central angle surrounding the center area is 180degrees. Alternatively, the first area, the center area, and the secondarea can be a strip shape elongating along a second direction.

According to the imaging device in the invention with various preferableembodiments described above, the first area and the second area areconstituted by polarizers, and the direction of an electric field of thefirst area passing light and the direction of an electric field of thesecond area passing light can be configured to be orthogonal to eachother. In addition, according to the imaging device of the inventionwith such a configuration, the direction of the electric field of thefirst area passing light can be configured to be parallel with the firstdirection, or the direction of the electric field of the first areapassing light can be configured to form the angle of 45 degrees with thefirst direction. Furthermore, in the imaging device of the inventionincluding an arbitrary combination of such configurations, the directionof the electric field of the first area passing light and the directionof an electric field of the third area passing light can be parallelwith each other, and the direction of the electric field of the secondarea passing light and the direction of an electric field of the fourtharea passing light can be parallel with each other. Moreover, in theimaging device of the invention including an arbitrary combination ofsuch configurations, it is desirable that an extinction ratio of thepolarizers is 3 or greater, and preferably 10 or greater.

Herein, a “polarizer” refers to a device that converts natural light(non-polarized light) or circularly-polarized light intolinearly-polarized light, and polarizers constituting the first area andthe second area themselves may be polarizers (polarizing plate) with aknown configuration and structure. In addition, for example, apolarization component of one of the first area passing light or thesecond area passing light may mostly be set to an S-wave (TE wave), anda polarization component of the other one of the first area passinglight or the second area passing light may mostly be set to a P-wave (TMwave). A polarization state of the first area passing light and thesecond area passing light may be linear polarization, or circularpolarization (however, rotation directions of the light are opposed toeach other). Generally, a horizontal wave of which the oscillatingdirection is only a specific direction is called a polarized wave, andthe oscillating direction is called a polarization direction or apolarization axis. The direction of an electric field of light coincideswith the polarization direction. An extinction ratio is a ratio betweena light component of which the direction of the electric field is thefirst direction and a light component of which the direction of theelectric field is the second direction included in light passing thefirst area in the first area, and a ratio between a light component ofwhich the direction of the electric field is the second direction and alight component of which the direction of the electric field is thefirst direction included in light passing the second area in the secondarea when the direction of the electric field of the first area passinglight is parallel with the first direction. In addition, when thedirection of the electric field of the first area passing light isconfigured to form the angle of 45 degrees with the first direction, theextinction ratio is a ratio between a light component of which thedirection of the electric field forms the angle of 45 degrees with thefirst direction and a light component of which the direction of theelectric field forms the angle of 135 degrees with the first directionincluded in the light passing the first area in the first area, and aratio between a light component of which the direction of the electricfield forms the angle of 135 degrees with the first direction and alight component of which the direction of the electric field forms theangle of 45 degrees with the first direction included in the lightpassing the second area in the second area. Alternatively, for example,when a polarization component of the first area passing light is mostlythe P-wave and a polarization component of the second area passing lightis mostly the S-wave, the extinction ratio is a ratio between a Ppolarization component and an S polarization component included in thefirst area passing light in the first area, and a ratio between an Spolarization component and a P polarization component included in thesecond area passing light in the second area.

In the imaging device of the invention with the various embodiments andconfigurations described above, an imaging element composing the firstimaging element group is composed of a photoelectric conversion element,and a color filter, an on-chip lens, and a wire-grid polarizer stackedon or above the element, and the wire-grid polarizer can be configuredto constitute the third area or the fourth area. Alternatively, theimaging element composing the first imaging element group is composed ofa photoelectric conversion element, and a wire-grid element, a colorfilter, and an on-chip lens stacked on or above the element, and thewire-grid polarizer can be configured to constitute the third area orthe fourth area. Alternatively, the imaging element is composed of aphotoelectric conversion element, and an on-chip lens, a color filter,and a wire-grid polarizer stacked on or above the element, and thewire-grid polarizer can be configured to constitute the third area orthe fourth area. However, the stacking order of the on-chip lens, thecolor filter, and the wire-grid polarizer can be appropriately changed.In addition, in these embodiments, when the direction of the electricfield of the first area passing light is parallel with the firstdirection, the direction in which a plurality of wires composing thewire-grid polarizer extends can be parallel with the first direction orthe second direction. Specifically, in a wire-grid polarizerconstituting the third area, the direction in which wires extend isparallel with the second direction, and in a wire-grid polarizerconstituting the fourth area, the direction in which wires extend isparallel with the first direction. Alternatively, in such embodiments,when the direction of the electric field of the first area passing lightforms the angle of 45 degrees with the first direction, the direction inwhich the plurality of wires constituting a wire-grid polarizer extendscan form the angle of 45 degrees with the first direction or the seconddirection. To be more specific, in the wire-grid polarizer constitutingthe third area, the direction in which the wires extend forms the angleof 135 degrees with the first direction and in the wire-grid polarizerconstituting the fourth area, the direction in which the wires extendforms the angle of 45 degrees with the first direction. The direction inwhich the wires extend is a light absorbing axis in the wire-gridpolarizer, and the direction orthogonal to the direction in which thewires extend is a light transmitting axis in the wire-grid polarizer.Furthermore, an imaging element composing the second imaging elementgroup can be configured or structured to be the same as the imagingelement composing the first imaging element group except that awire-grid polarizer is not provided.

In the imaging device of the invention with the various preferableembodiments and configuration described above, the imaging element arrayhas a Bayer arrangement, and one pixel can be composed of four imagingelements. In addition, in a first pixel group, one third area and/orfourth area can be arranged for one pixel. In other words, an embodimentwhere one third area is arranged, one fourth area is arranged, or onethird area and one fourth area are arranged for one pixel can beconfigured. Alternatively, in the imaging device of the invention withthe various embodiments and configuration described above, it can beconfigured such that the first pixel group is constituted by two unitpixel rows, the third area is arranged in one of the unit pixel row, andthe fourth area is arranged in the other one of the unit pixel.Alternatively, it can be configured such that the first pixel group isconstituted by one unit pixel row, and the third area and the fourtharea are arranged in the one unit pixel row. However, the arrangement ofthe imaging element array is not limited to a Bayer arrangement, andother arrangements such as an interline arrangement, a G-striped andRB-checkered arrangement, a G-striped and RB-complete-checkeredarrangement, a checkered complementary-color arrangement, a stripearrangement, an oblique-stripe arrangement, a primary-colorcolor-difference arrangement, a field color-difference sequencearrangement, a frame color-difference sequence arrangement, an MOSarrangement, a modified MOS arrangement, a frame interleavedarrangement, and a field interleaved arrangement can be exemplified.

Alternatively, when the arrangement of the imaging element array is setto a Bayer arrangement in the first pixel group, in one pixel, a redimaging element sensing red and a blue imaging element sensing blue arenot arranged with the third area and the fourth area, but one of twogreen imaging elements sensing green may be arranged with the thirdarea, and the other may be arranged with the fourth area. Alternatively,when the arrangement of the imaging element array is set to a Bayerarrangement in the first pixel group, in one pixel, two imaging elements(for example, one red imaging element sensing red and one of two greenimaging elements sensing green) adjacent to the first direction amongthe red imaging element sensing red, one blue imaging element sensingblue, and the two green imaging elements sensing green may be arrangedwith the third area or the fourth area, and remaining two imagingelements (for example, the blue imaging element sensing blue and theother green imaging element sensing green) may be arranged with thefourth area or the third area. Alternatively, when the arrangement ofthe imaging element array is set to a Bayer arrangement in the firstpixel group, in one pixel, any one imaging element (for example, one redimaging element sensing red or one blue imaging elements sensing blue)among the one red imaging element sensing red, the one blue imagingelement sensing blue, and two green imaging elements sensing green maybe arranged with the third area or the fourth area, and an imagingelement (for example, the green imaging element) adjacent to the seconddirection among the imaging elements may be arranged with the fourtharea or the third area.

The number of unit pixel rows constituting the first pixel group can beexemplified as one or two as described above, but is not limitedthereto. Imaging elements constituting a pixel in the first pixel groupare set to the first imaging element group, but imaging elementsconstituting all pixels in the first pixel group may be set to the firstimaging element group, and imaging elements constituting a part of thepixels in the first pixel group may be the first imaging element group.In addition, an imaging element group constituted by imaging elementsnot included in the first imaging element group is set to the secondimaging element group, but an imaging element group constituted by allimaging elements not included in the first imaging element group may beset to the second imaging element group.

In the imaging method of the invention with the various preferableembodiments and configurations described above (hereinbelow, it may becollectively referred to simply as “the present invention”), the firstdirection can be set to a horizontal direction and the second directioncan be set to a vertical direction. In the first pixel group, unitlengths of the third area and the fourth area along the first directionmay be, for example, equivalent to the length of imaging elements alongthe first direction (when the direction of the electric field of thefirst area passing light is in parallel with the first direction), ormay be equivalent to the length of one imaging element (when thedirection of the electric field of the first area passing light formsthe angle of 45 degrees with the first direction). The lens system mayinclude a single-focus lens, or a so-called zoom lens, and theconfiguration or structure of a lens or the lens system may bedetermined based on a specification required for the lens system. As animaging element, signal-amplifying image sensor such as a CCD (ChargeCoupled Device) element, a CMOS (Complementary Metal OxideSemiconductor) image sensor, a CIS (Contact Image Sensor), and a CMD(Charge Modulation Device) can be exemplified. In addition, as theimaging device, a surface irradiation type solid-state imaging device ora rear surface irradiation type solid-state imaging device can beexemplified. Furthermore, for example, a digital still camera, a videocamera, or a camcorder can be constituted by the imaging device and thelike of the present invention. In addition, the imaging method of theinvention can be applied to the technology disclosed inJP-A-2004-309868, or the like.

When the third area and the fourth area are constituted by a wire-gridpolarizer, it is preferable that wires constituting the wire-gridpolarizer are not limited, but formed of aluminum (Al) or an aluminumalloy, the value of a ratio of the width of a wire to the pitch of awire [(the width of a wire)/(the pitch of a wire)] is 0.33 or greater,the height of a wire is

-   5×10⁻⁸ m

or greater, and the number of wires is 10 or more.

In the invention, the barycenter of the first area refers to thebarycenter obtained based on the external shape of the first area, andthe barycenter of the second area refers to the barycenter obtainedbased on the external shape of the second area. When the external shapeof the first polarization means is set to be a circular shape with theradius r, and the first area and the second area are respectively set tobe a semilunar shape that occupies half of the first polarization means,the distance between the barycenter of the first area and the barycenterof the second area can be obtained from a simple calculation of[(8r)/(3π)]−

Embodiment 1

Embodiment 1 relates to the imaging method of the invention, and morespecifically to an imaging method for imaging a subject as astereoscopic image.

A conceptual diagram of the imaging device of the invention appropriatefor the execution of the imaging method of Embodiment 1 is shown in (A)of FIG. 1, polarization states in the first polarization means and thesecond polarization means are schematically shown in (B) and (c) of FIG.1, a conceptual diagram of the light that passes through the lenssystem, the first area in the first polarization means and the thirdarea in the second polarization means and reaches the imaging elementarray is shown in (A) of FIG. 2, a conceptual diagram of light thatpasses the second area in the first polarization means and the fourtharea in the second polarization means and reaches the imaging elementarray is shown in (B) of FIG. 2, images formed in the imaging elementarray by the light shown in (A) and (B) of FIG. 2 is schematically shownin (C) and (D) of FIG. 2. Furthermore, in the description below, thelight advancing direction is set to the Z-axis direction, the firstdirection to the horizontal direction (X-axis direction), and the seconddirection to the vertical direction (Y-axis direction). In addition, aconceptual diagram of the imaging element array with a Bayer arrangementin the imaging device of Embodiment 1 is shown in FIG. 4.

In the imaging device in Embodiment 1 or Embodiments 2 to 6 to bedescribed below, unit pixel rows composed of M₀ (for example, 1920 inEmbodiment 1) pixels along the first direction (horizontal direction orX-axis direction) are arranged in N₀ (for example, 1080 in Embodiment 1)rows along the second direction (vertical direction or Y-axis direction)orthogonal to the first direction, and the device includes (A) theoptical system, and (B) an imaging element array 40 in which imagingelements 43A and 43B are arranged corresponding to each pixel and whichconverts light passing through the optical system into electric signals.Furthermore, the values of M₀ and N₀ are essentially arbitrary, and notlimited to the values above.

Herein, a pixel group composed of at least one unit pixel row includinga unit pixel row selected for every N-th row

-   (where 2≦N)

is set to a first pixel group PG₁, imaging elements composing a pixel inthe first pixel group PG₁ are set to a first imaging element group 41,an imaging element group composed of imaging elements not included inthe first imaging element group 41 is set to a second imaging elementgroup 42, and a pixel group constituted by pixels composed of the secondimaging element group 42 is set to a second pixel group PG₂.

Furthermore, in the imaging device of Embodiment 1, N=2^(n), and n is anatural number from 1 to 5, and more specifically, n=3.

In Embodiment 1, or Embodiments 2 to 6 to be described later, theoptical system includes (a) first polarization means 130, 230, and 330that polarize light from a subject, and (b) a lens system 20 thatconverges light from the first polarization means 130, 230, and 330. Inaddition, the first imaging element group 41 has second polarizationmeans 150 and 250 in the side of light incidence, converts lightconverged by the lens system 20 into electric signals, and the secondimaging element group 42 converts the light converged by the lens system20 into electric signals. Specifically, the first imaging element group41 converts light that is converged by the lens system 20 and passes thefirst polarization means 130, 230, and 330 and the second polarizationmeans 150 and 250 into electric signals. The second imaging elementgroup 42 converts light that is converged by the lens system 20 andpasses the first polarization means 130, 230, and 330 into electricsignals. The first polarization means 130, 230, and 330 has first areas131, 231, and 331 and second areas 132, 232, and 332 arranged along thefirst direction (horizontal direction or X-axis direction).

Furthermore, a polarization state of a first area passing light L₁passing the first areas 131, 231, and 331 and a polarization state of asecond area passing light L₂ passing the second areas 132, 232, and 332are different from each other, the second polarization means 150 and 250has a plurality of third areas 151 and 251 and fourth areas 152 and 252extending in the first direction (horizontal direction or X-axisdirection), a polarization state of a third area passing light L₃passing the third areas 151 and 251 and a polarization state of a fourtharea passing light L₄ passing the fourth areas 152 and 252 are differentfrom each other, the first area passing light L₁ passes the third areas151 and 251 and then reaches the first imaging element group 41, thesecond area passing light L₂ passes the fourth areas 152 and 252 andthen reaches the first imaging element group 41, and accordingly,parallax information is obtained in which the distance between thebarycenter BC₁ of the first areas 131, 231, and 331 and the barycenterBC₂ of the second areas 132, 232, and 332 is set to the base-line lengthof the parallax of both eyes.

In the imaging device of Embodiment 1 or Embodiments 2 to 6 to bedescribed later, the lens system 20 includes, for example, a capturinglens 21, a diaphragm 22, and an imaging lens 23, and functions as a zoomlens. The capturing lens 21 is a lens for converging light incident froma subject. The capturing lens 21 includes a focus lens for taking thefocus, a zoom lens for magnifying a subject, or the like, and generally,is realized by a combination of a plurality of lenses for correctingchromatic aberration or the like. The diaphragm 22 has a function ofnarrowing down in order to adjust the amount of converged light, andgenerally, is configured to be combined with a plurality of plate-likeblades. At least in the position of the diaphragm 22, light from onepoint of a subject is parallel light. The imaging lens 23 forms an imageon the imaging element array 40 with light passing the firstpolarization means 130, 230, and 330. The imaging element array 40 isarranged inside a camera main body 11. In the above configuration, anentrance pupil is positioned more in the camera main body side than inthe imaging lens 23 side. The imaging device constitutes, for example, adigital still camera, a video camera, or a camcorder.

The camera main body 11 includes, for example, an image processing means12 and an image storage unit 13, in addition to the imaging elementarray 40. In addition, right-eye parallax information, left-eye parallaxinformation, and image information are generated based on the electricsignals converted by the imaging element array 40. The imaging elementarray 40 is realized by, for example, a CCD element, a CMOS imagesensor, or the like. The image processing means 12 finally creates theparallax information and the image information from the electric signalsoutput from the imaging element array 40 and records in the imagestorage unit 13.

The first polarization means 130, 230, and 330 are arranged around thediaphragm 22 of the lens system 20. Specifically, the first polarizationmeans 130, 230, and 330 are arranged on positions around the diaphragm22, if possible, as long as the means do not hinder the operation of thediaphragm 22. Furthermore, the first polarization means 130, 230, and330 are arranged near the lens system 20 in the state of parallel lightwhen light incident to the lens system 20 is parallel light first andfinally converged (forms an image) on the imaging elements 43A and 43Bas described above.

In the imaging device 110 of Embodiment 1, the first polarization means130 includes the first area 131 and the second area 132 arranged alongthe first direction. Specifically, the external shape of the firstpolarization means 130 is a circular shape, and the first area 131 andthe second area 132 each have semilunar external shapes occupying halfof the first polarization means 130. The boundary line between the firstarea 131 and the second area 132 extends along the second direction. Thefirst polarization means 130 constituted by the combination of twopolarization filters separates incident light into two differentpolarization states. The first polarization means 130 is composed ofbilaterally symmetric polarizers as described above, and generatespolarized light beams in the linear direction orthogonal to each otheror polarized light beams in the rotation direction opposed to each otherat two left and right positions in the erecting state of a camera. Thefirst area 131 is a filter that performs polarization for an image ofthe subject that the right eye is supposed to see (light that the righteye is supposed to receive) in the first pixel group PG₁. On the otherhand, the second area 132 is a filter that performs polarization for animage of the subject that the left eye is supposed to see (light thatthe left eye is supposed to receive) in the first pixel group PG₁.

Herein, in the imaging device 110 of Embodiment 1, the first area 131and the second area 132 are constituted by polarizers. In addition, thedirection of the electric field of the first area passing light L₁(indicated by a white arrow) and the direction of the electric field ofthe second area passing light L₂ (indicated by a white arrow) areorthogonal to each other (refer to (B) of FIG. 1). Herein, in Embodiment1, the direction of the electric field of the first area passing lightL₁ is in parallel with the first direction. Specifically, for example,the first area passing light L₁ mostly has a P-wave (TM wave) as apolarization component and the second area passing light L₂ mostly hasan S-wave (TE wave) as a polarization component. Furthermore, thedirection of the electric field of the first area passing light L₁ anddirection of the electric field of the third area passing light L₃(indicated by white arrows) are in parallel with each other, and thedirection of the electric field of the second area passing light L₂ anddirection of the electric field of the fourth area passing light L₄(indicated by white arrows) are in parallel with each other (refer to(C) of FIG. 1). In addition, the extinction ratio of each polarizer is 3or greater, and more specifically 10 or greater.

In the imaging device 110 of Embodiment 1, the external shape of thefirst polarization means 130 is a circular shape with the radius of r=10mm. In addition, the first area 131 and the second area 132 havesemilunar shapes occupying half of the first polarization means 130.Therefore, the distance between the barycenter BC₁ of the first area 131and the barycenter BC₂ of the second area 132 is[(8r)/(3π)]=8.5 mm.

As a schematic partial cross-sectional diagram is shown in (A) of FIG.3, and the arrangement state of the wire-grid polarizer 67 isschematically shown in (B) of FIG. 3, the imaging element 43A composingthe first imaging element group 41 is composed of, for example, aphotoelectric conversion element 61 provided on a silicon semi-conductorsubstrate 60, and a first flattening film 62, a color filter 63, anon-chip lens 64, a second flattening film 65, an inorganic insulatingbase layer 66, and the wire-grid polarizer 67 stacked on thephotoelectric conversion element 61. In addition, the wire-gridpolarizer 67 is constituted by the third area 151 and the fourth area152. Furthermore, in (B) of FIG. 3, solid lines indicate boundary areasof a pixel, and dotted lines indicate boundary areas of the imagingelement 43A. The direction in which a plurality of wires 68 composingthe wire-grid polarizer 67 extends is in parallel with the firstdirection or the second direction. Specifically, in a wire-gridpolarizer 67A constituting the third area 151, the direction in which awire 68A extends is in parallel with the second direction, and in awire-grid polarizer 67B constituting the fourth area 152, the directionin which a wire 68B extends is in parallel with the first direction. Thedirection in which the wire 68 extends is a light absorption axis in thewire-grid polarizer 67, and the direction orthogonal to the direction inwhich the wire 68 extends is a light transmission axis in the wire-gridpolarizer 67. The imaging element 43B composing the second imagingelement group 42 can be configured or structured to be the same as theimaging element 43A composing the first imaging element group 41 exceptthat a wire-grid polarizer is not provided therein.

As schematically shown in (A) and (B) of FIG. 2, a square-shaped objectA is assumed to come into focus of the lens system 20. In addition, acircular-shaped object B is assumed to be positioned closer to the lenssystem 20 than the object A. The image of the square-shaped object A isformed on the imaging element array 40 in the state of coming intofocus. In addition, the image of the circular-shaped object B is formedon the imaging element array 40 in the state of not coming into focus.Then, in the example shown in (A) of FIG. 2, the image of the object Bis formed on a position a distance

-   (+ΔX)

apart from the object A to the right side on the imaging element array40. On the other hand, in the example shown in (B) of FIG. 2, the imageof the object B is formed on a position a distance

-   (−ΔX)    apart from the object A to the left side on the imaging element    array 40. Therefore, the distances-   (2×ΔX)    are information regarding the depth of the object B. In other words,    the blurring amount and blurring direction of an object positioned    closer to the imaging device than the object A are different from    the blurring amount and blurring direction of another object    positioned away from the imaging device, and the blurring amount of    the object B is different according to the distance between the    object A and object B. In addition, it is possible to obtain a    stereoscopic image by setting the distance between the barycentric    positions in the shapes of the first area 131 and the second area    132 in the first polarization means 130 to the base-line length of    parallax of both eyes. In other words, it is possible to obtain a    stereoscopic image from right-eye parallax information (refer to the    schematic diagram of (C) of FIG. 2) and left-eye parallax    information (refer to the schematic diagram of (D) of FIG. 2)    obtained in the first pixel group PG₁ as above.

A conceptual diagram of the imaging element array with a Bayerarrangement in the imaging device of Embodiment 1 is shown in FIG. 4.Herein, one pixel is composed of four imaging elements (one red imagingelement R that senses red, one blue imaging element B that senses blue,and two green imaging elements G that sense green). In addition, a pixelgroup constituted by at least one unit pixel row (two unit pixel rows inEmbodiment 1) including a unit pixel row selected for every N-th row(where

-   2≦N,

and in Embodiment 1, N=8 as described above) is set to the first pixelgroup PG₁. In other words, in Embodiment 1, the third area 151 isdisposed in one unit pixel row arranged along the first direction, andthe fourth area 152 which is adjacent to the unit pixel row in thesecond direction is disposed in one unit pixel row arranged along thefirst direction. In the first pixel group PG₁, for one pixel, one thirdarea 151 or fourth area 152 is disposed, the third area 151 is disposedfor all pixels composing one unit pixel row, and the fourth area 152 isdisposed for all pixels composing one unit pixel row. In other words,imaging elements composing all pixels in the first pixel group PG₁ isset to the first imaging element group 41. Furthermore, the third area151 and the fourth area 152 extend in the first direction as a whole,but a unit length extending in the first direction and the seconddirection of the third area 151 and the fourth area 152 is equal to alength along the first direction and the second direction of the imagingelement 43A. In addition, by adopting such a configuration, astrip-shaped image extending in the first direction based on lightmostly having P-wave components (right-eye parallax information) and astrip-shaped image extending in the first direction based on lightmostly having S-wave components are generated along the seconddirection. Furthermore, in FIG. 4, a vertical line is drawn in the thirdarea 151 and vertical and horizontal lines are drawn in the fourth area152, but they schematically indicate wires of the wire-grid polarizer67A and 67B.

In addition, in the imaging method of Embodiment 1 or in Embodiments 2to 6 to be described later, parallax information for obtaining astereoscopic image in the first imaging element group 41 is acquired,image information for obtaining images in the second imaging elementgroup 42 is acquired, image information is obtained in pixels from whichparallax information is acquired in the first pixel group PG₁(specifically, all pixels in the first pixel group PG₁ in Embodiment 1)based on the acquired image information, and then a stereoscopic imageis obtained from the parallax information and the image information forall pixels.

In other words, in the first pixel group PG₁, a depth map (depthinformation) is acquired as parallax information based on the amount ofparallax generated from electric signals obtained by the first areapassing light passing the third area 151 and by the second area passinglight passing the fourth area 152. In addition, image information isacquired based on electric signals from all remaining pixel elements 43B(the second imaging element group 42) constituting the imaging elementarray 40. Such acquisition and processing methods can adopt knownmethods.

Since the image information and amount of light obtained from eachimaging element in the first pixel group PG, constituted by the firstimaging element group 41 where the third area 151 and the fourth area152 are disposed (hereinbelow, collectively referred to as imageinformation) is image information acquired from light passing an areaseparated into the first area 131 and the second area 132 for acquiringeach parallax information, it is not possible to obtain the same imageinformation as the image information of each imaging element in thesecond imaging element group 42 acquired from light obtained by addingthe first area passing light passing the first area 131 and the secondarea passing light passing the second area 132 without separating theparallax information. For that reason, it is necessary to obtaininsufficient or lacking image information based on the image informationfrom each imaging element constituting the second imaging element group42 of the adjacent first pixel group for each imaging element from whichparallax information is acquired in the first pixel group PG₁constituted by the first imaging element group 41 where the third area151 and the fourth area 152 are disposed. In other words, imageinformation is generated which is the same as the second imaging elementgroup based on an interpolation process for a unit pixel row includingthe first imaging element group where the third area 151 and the fourtharea 152 are disposed. By synthesizing with the addition of the imageinformation in the first imaging element group and the image informationin the second imaging element group obtained as above, it is possible toobtain image information without insufficiency or lacking in the entireimaging elements. In addition, it is possible to make parallaxemphasized or appropriate with, for example, a parallax detectiontechnology for creating a disparity map by performing stereo-matchingfrom a difference between left-eye parallax information obtained fromthe third area and right-eye parallax information obtained from thefourth area, and a parallax control technology for arbitrarilygenerating left-eye images and right-eye images based on the obtaineddisparity map with image information of the entire imaging elementsobtained by the addition of the first imaging element group and thesecond imaging element group based on an interpolation process.

Specifically, electric signals for obtaining right-eye parallaxinformation are generated in the imaging element 43A by the first areapassing light L₁ that passes the third area 151 and reaches the imagingelement 43A. In addition, electric signals for obtaining left-eyeparallax information are generated in the imaging element 43A by thesecond area passing light L₂ that passes the fourth area 152 and reachesthe imaging element 43A. Then, both electric signals are output at thesame time or alternately in a time series. On the other hand, electricsignals for obtaining image information (two-dimensional imageinformation) are generated in the imaging element 43B and output bylight that passes the first area 131 and the second area 132 and reachesthe imaging element 43B. An imaging process is performed for the outputelectric signals (electric signals for obtaining the right-eye parallaxinformation, left-eye parallax information, and image information outputfrom the imaging element array 40) by the image processing means 12, andrecorded in the image storage unit 13 as parallax information and imageinformation.

FIG. 5 shows a conceptual diagram of the imaging element array thatperforms a demosaicing process for the electric signals obtained fromthe imaging element 43B constituting the second imaging element group 42and has a Bayer arrangement for describing an imaging process forobtaining signal values. Furthermore, FIG. 5 shows an example in whichsignal values relating to a green imaging element are generated. In ageneral demosaicing process, an average value of electric signals ofneighboring imaging elements with the same color is commonly used.However, when a unit pixel row for obtaining right-eye parallaxinformation and a unit pixel row for obtaining left-eye parallaxinformation are disposed adjoining each other as in Embodiment 1, thereis a concern that the original image information is not obtained ifneighboring values are used without change. Thus, the demosaicingprocess is performed in order to prevent such a problem. It is possibleto obtain imaging element signal values in each imaging element positionby the demosaicing process, but the stage may be put in a kind of astate of omission as described above. In other words, in the first pixelgroup PG₁ where the first imaging element group 41 is disposed, the sameimage information as that from the second imaging element group 42 isnot obtained. For that reason, it is necessary to generate imagingelement signal values by interpolation for an area where imaging elementsignal values do not exist (the first imaging element group 41). As aninterpolation method, known methods such as a method using an average ofaddition of neighboring values or the like can be exemplified.Furthermore, the interpolation process may be performed in parallel withthe demosaicing process. Since the number of pixels is completelymaintained in the first direction, deterioration in image quality suchas a decline in resolution of an overall image is relativelyinsignificant. In addition, accordingly, it is possible to obtain imageinformation in pixels from which parallax information is acquired in thefirst pixel group PG₁ (to be more specific, all pixels in the firstpixel group PG₁ in Embodiment 1).

In a Bayer arrangement, the red imaging element R is assumed to bedisposed in a position (4,2). At this point, an arithmetic operationexpressed by the following formula is performed in order to generate agreen imaging element signal value g′ corresponding to the position(4,2).g′ _(4,2)=(g _(4,1) +g _(4,3) +g _(5,2) +g _(1,2) ×W ₃)/(3.0+W ₃)

Where g′_(i,j) in the left side is a green imaging element signal valuein a position (i, j). In addition, g_(i,j) in the right side is a valueof an electric signal of the green imaging element in the position (i,j). Furthermore, when a distance(W₁) from the target imaging elementG_(4,2) to neighboring imaging elements G_(4,1), G_(4,3), and G_(5,2) iseach set to, for example, “1.0”, “3.0” is a value obtained such thatinverses thereof are set to weights, and the weights are summed. In thesame manner, W₃ is a weight for an electric signal value of the imagingelement G_(1,2) separated for three imaging elements, and the value is“⅓” in this case. If the above formula is generalized, it turns into thefollowing formula.

When i is an even number (a signal value of the green imaging element Gcorresponding to the position of the red imaging elementg′ _(i,j)=(g _(i,j−1) ×W ₁ +g _(i,j+1) ×W ₁ +g _(i+1,j) ×W ₁ +g _(i−3,j)×W ₃)/(W ₁×3.0+W ₃),  R):

and when i is an odd number (a signal value of the green imaging elementG corresponding to the position of the blue imaging elementg′ _(i,j)=(g _(i,j−1) ×W ₁ +g _(i,j+1) ×W ₁ +g _(i−1,j) ×W ₁ +g _(i+3,j)×W ₃)/(W ₁×3.0+W ₃),  B):

where W₁=1.0 and W₃=⅓.

It is possible to perform the demosaicing process for the red imagingelement R and the blue imaging element B in the same manner.

It is possible to obtain imaging element signal values in each imagingelement position by the demosaicing process, but the stage may be put ina kind of a state of omission as described above. In other words, in thefirst pixel group PG₁ where the first imaging element group 41 isdisposed, image information the same as that from the second imagingelement group 42 is not obtained. For that reason, it is necessary togenerate imaging element signal values by interpolation for an areawhere imaging element signal values do not exist (the first imagingelement group 41). As an interpolation method, known methods such as amethod using an average of addition of neighboring values or the likecan be exemplified. Furthermore, the interpolation process may beperformed in parallel with the demosaicing process. Since the imagequality is completely maintained in the first direction, deteriorationin image quality such as a decline in resolution of an overall image isrelatively insignificant. In addition, accordingly, it is possible toobtain image information in pixels from which parallax information isacquired in the first pixel group PG₁ (to be more specific, all pixelsin the first pixel group PG₁ in Embodiment 1).

In addition, a stereoscopic image is obtained from the obtained parallaxinformation and the image information in all pixels. In other words,after right-eye image data and left-eye image data are obtained from theobtained parallax information and the image information in all pixels, astereoscopic image is displayed based on the right-eye image data andthe left-eye image data. Furthermore, such a processing method itselfcan adopt a known one.

In the imaging method of Embodiment 1, parallax information forobtaining a stereoscopic image is acquired in the first imaging elementgroup, image information for obtaining an image is acquired in thesecond imaging element group, image information is obtained in a pixelfrom which the parallax information is acquired in the first pixel groupbased on the acquired image information, and then a stereoscopic imageis obtained from the parallax information and the image information forall pixels. In other words, since the image information for obtaining animage is basically acquired in the second imaging element group, that isto say, light passing the first area and the second area of the firstpolarization means is incident to the second imaging element group in amixed state, image information can be obtained by non-polarized light,and as a result, the occurrence of visual field competition can besuppressed. Moreover, since the parallax information for obtaining astereoscopic image is acquired in some of the pixels, in other words, inthe first imaging element group, it is possible to prevent a drasticdrop in the amount of light that reaches the imaging element array, incomparison with the case where the parallax information for obtaining astereoscopic image is acquired in all pixels. In other words, in termsof the intensity 100 of incident natural light, the amount of light thatpasses the first polarization means 130 and the second polarizationmeans 150 (light that reaches the first imaging element group) is about25% of the amount of light before being incident to the firstpolarization means 130 even if the transmittance loss is zero. On theother hand, the amount of light that passes the first polarization means130 (light that reaches the second imaging element group) does notchange from the amount of light that is incident to the firstpolarization means 130 even if the transmittance loss is zero. For thatreason, it is possible to prevent a drastic drop in the amount of lightthat reaches the overall imaging element array.

Furthermore, with regard to the image quality and number of pixels of animage, the ratio of the image quality and the number of pixels of thedepth map is not set to 1:1, but this is because individual subjects aresufficiently large in comparison with pixel resolution power in mostcaptured scenes, and the same distance information resolution power asthe pixel resolution power of an image is not necessary for individualsubjects as long as there is no distance difference in the same finenessas the pixel resolution power. In addition, if resolution power in thehorizontal direction is sufficient in sensing the distance difference,there is little discomfort even when resolution power in the verticaldirection is low.

In addition, in Embodiment 1, since the imaging device 110 isconstituted by one pair of the first polarization means 130 and thesecond polarization means 150, and one lens system 20, two differentimages separated, for example, to the left and right can be generated atthe same time, and it is possible to provide an imaging device that issmall and monocular, and has a simple configuration and structure, and asmall number of constituent components. In addition, since two pairs ofthe combination of a lens and a polarization filter are not necessary,no deviations or differences occur in zoom, diaphragms, focus,convergence angle, or the like. Moreover, since the base-line length ofthe parallax of both eyes is relatively short, a natural stereoscopiceffect can be obtained. Furthermore, two-dimensional images orthree-dimensional images can be easily obtained by adopting a structureresulting from attaching or detaching the first polarization means 130.

Embodiment 2

Embodiment 2 is a modification of Embodiment 1. In Embodiment 1, thedirection of the electric field of the first area passing light L₁ isset to be in parallel with the first direction. On the other hand, inEmbodiment 2, the direction of the electric field of the first areapassing light L₁ is set to form the angle of 45 degrees with the firstdirection. In addition, the direction of the electric field of the firstarea passing light L₁ and the direction of the electric field of thethird area passing light L₃ are in parallel with each other, and thedirection of the electric field of the second area passing light L₂ andthe direction of the electric field of the fourth area passing light L₄are in parallel with each other. States of polarization in the firstpolarization means 230 and a second polarization means 250 provided inan imaging device of Embodiment 2 are schematically shown in (A) and (B)of FIG. 6.

A conceptual diagram of the imaging element array 40 with a Bayerarrangement is shown in FIG. 7. Also in Embodiment 2, one pixel of theimaging element array 40 is constituted by four imaging elements (onered imaging element R that senses red, one blue imaging element B thatsenses blue, and two green imaging elements G that sense green). Inaddition, in the first pixel group PG₁, a third area 251 is disposed inone unit pixel row arranged along the first direction, and a fourth area252 which is adjacent to the unit pixel row in the second direction isdisposed in one unit pixel row arranged along the first direction. Thethird area 251 and the fourth area 252 are disposed along the seconddirection for every N-th row. Furthermore, the third area 251 and thefourth area 252 extend in the first direction as a whole, but the unitlength of the third area 251 and the fourth area 252 is equivalent tothe length of one imaging element. In addition, with such aconfiguration, a strip-shaped image extending in the first directionbased on light mostly with P-wave components (right-eye parallaxinformation) and a strip-shaped image extending in the first directionbased on light mostly with S-wave components are generated along thesecond direction. Furthermore, in FIG. 7, oblique lines are drawn insidethe third area 251 and the fourth area 252, but they schematicallyindicate wires of a wire-grid polarizer.

Except for the above points, as an imaging method using the imagingdevice of Embodiment 2 can be the same as that described in Embodiment1, detailed description thereof will be omitted. In addition, as theconfiguration and structure of the imaging device of Embodiment 2 arethe same as those of the imaging device 110 described in Embodiment 1,detailed description thereof will be omitted. The configuration andstructure of the imaging device of Embodiment 2 can be applied toimaging devices in Embodiments 3 to 6 to be described below.

Embodiment 3

Embodiment 3 is also a modification of Embodiment 1. In a firstpolarization means 330 of an imaging device of Embodiment 3, a centerarea 333 is provided between a first area 331 and a second area 332, anda polarization state of a center area passing light that passes thecenter area 333 does not change from a state before being incident tothe center area 333. In other words, the center area 333 is afree-passing state of the polarization.

Incidentally, when incident light passes the first polarization means,the amount of light decreases in proportion to spectral characteristicsand extinction ratios, and the brightness becomes darker. Herein, theextinction ratio refers to a ratio of the amount of passing light withthe selection of a polarizer and the amount of leaking light byreflection or absorption without the selection of a polarizer.Specifically, for example, in the case of a polarizer that makes P-wavecomponents with the extinction ratio of 10 pass therethrough, for theintensity 100 of incident natural light with P-wave components: S-wavecomponents=50:50, the polarizer transmits light at a ratio of 50 of theP-wave components and 5 of the S-wave components. In addition, in thecase of a polarizer that makes P-wave components with the extinctionratio of 1 pass therethrough, the P-wave components are transmitted100%, but the S-wave components are not transmitted but are allreflected or completely absorbed, and therefore, when average naturallight is incident, the brightness becomes about ½. The amount of lightthat passes the first polarization means 130 and the second polarizationmeans 150 shown in (B) and (C) of FIG. 1 is about 25% of the amount oflight before being incident to the first polarization means 130 even ifthe transmission loss is zero. In addition, when the light that passesthe first and the second areas is in a mixed state and incident to theimaging element array 40 in an inseparable state, the base-line lengthof parallax of both eyes becomes shorter in proportion to the mixedratio, and left-eye parallax information and right-eye parallaxinformation become the same in a completely mixed state, parallax is nottaken, and stereoscopic view is not possible.

In the center area 333 of the first polarization means 330, the lightintensity is strong, but the parallax amount is small. Thus, it ispossible to increase the light intensity that the imaging element array40 receives and to secure a sufficient base-line length of parallax ofboth eyes by adopting the first polarization means 330 of Embodiment 3.As shown by the schematic diagram of the first polarization means 330 in(A) of FIG. 8, when the external shape of the first polarization means330 is a circular shape, the center area 333 can be a circular shape,and the first area 331 and the second area 332 can be a fan shape withthe center angle of 180 degrees surrounding the center area 333.Alternatively, as shown by the schematic diagrams of the firstpolarization means 330 in (B) and (C) of FIG. 8, the center area 333 canbe a rhombic or square shape, and the first area 331 and the second area332 can be a shape similar to a fan shape with the center angle of 180degrees surrounding the center area 333. Alternatively, as shown by theschematic diagram of the first polarization means 330 in (D) of FIG. 8,the first area 331, the center area 333, and the second area 332 can bestrip shapes extending along the second direction.

Except for the above points, as an imaging method using the imagingdevice of Embodiment 3 can be the same as that described in Embodiment1, detailed description thereof will be omitted. In addition, as theconfiguration and structure of the imaging device of Embodiment 3 can bethe same as those of the imaging device 110 described in Embodiment 1,detailed description thereof will be omitted. The configuration andstructure of the imaging device of Embodiment 3 can be applied toimaging devices in Embodiments 4 to 6 to be described below.

Embodiment 4

Embodiment 4 is also a modification of Embodiment 1. In Embodiment 4,the relationship between an extinction ratio and parallax is examined.In other words, a synthesized image simulation is preformed forexamining, when images separated to the left and right are mixed, if theparallax disappears, in other words, if stereoscopic view is notpossible to what extent the images should be mixed by changingextinction ratios from the extinction ratio=∞ (with 0%

crosstalk and in a state where left-eye parallax information andright-eye parallax information are completely separated) to theextinction ratio=1 (with 50% crosstalk, in a state where a left-eyeimage and a right-eye image are completely mixed, and left-eye parallaxinformation and right-eye parallax information are the same parallaxinformation (image)). A part of the result is shown in (A) and (B) ofFIG. 9.

Herein, (A) of FIG. 9 shows the state of the extinction ratio=\, and (B)of FIG. 9 shows the state of the extinction ratio=3 (with 25%crosstalk). In the diagrams in the left sides (left-eye images) anddiagrams in the right sides (right-eye images) of (A) and (B) of FIG. 9,distances between the solid lines and broken lines extending in thevertical direction are the same. When the diagrams in the left sides(left-eye images) and diagrams in the right sides (right-eye images) of(A) and (B) of FIG. 9 are compared, the positions of the nose of theplaster figure located to the rear side of the apple are slightlydifferent. In addition, when (A) and (B) of FIG. 9 are compared, thedifference in the positions of the nose of the plaster figure is smallin (B) of FIG. 9, in comparison with (A) of FIG. 9. Although not shownin the diagrams, when the extinction ratio=1, the positions of the noseof the plaster figure located in the rear side of the apple are same inthe left-eye images and the right-eye images. In addition, when theextinction ratio=10 (with 10% crosstalk), a difference in the positionsof the nose of the plaster figure is small in comparison with (A) ofFIG. 9 and large in comparison with (B) of FIG. 9. From the resultsabove, it is determined that the extinction ratio of a polarizer isdesirably 3 or greater.

Embodiment 5

Embodiment 5 is also a modification of Embodiment 1. In Embodiment 5,the relationship between specifications and extinction ratios of awire-grid polarizer is obtained from calculations. Specifically, therelationship between the pitch of the wire constituting a wire-girdpolarizer,

-   the wavelength (λ)

of incident light, and an extinction ratio is shown in (A) of FIG. 10.Furthermore, the width of a wire is set to ⅓ of the pitch of a wire, theheight of a wire to 150 nm, and the length of a wire to infinite. In (A)of FIG. 10, the curve “A” is data in the case of the pitch of 150 nm,the curve “B” is data in the case of the pitch of 175 nm, the curve “C”is data in the case of the pitch of 200 nm, the curve “D” is data in thecase of the pitch of 250 nm, and the curve “E” is data in the case ofthe pitch of 300 nm. In addition, the relationship between the height ofa wire constituting the wire-grid polarizer, the wavelength (l) ofincident light, and an extinction ratio is shown in (B) of FIG. 10.Furthermore, the width of a wire is set to 50 nm, the length of a wireto infinite, and the pitch of a wire to 150 nm. In (B) of FIG. 10, thecurve “A” is data in the case of the height of 250 nm, the curve “B” isdata in the case of the height of 200 nm, the curve “C” is data in thecase of the height of 150 nm, and the curve “D” is data in the case ofthe height of 100 nm. In addition, the relationship between the(width/pitch) of a wire constituting the wire-grid polarizer, thewavelength (l) of incident light, and an extinction ratio is shown in(C) of FIG. 10. Furthermore, the width of a wire is set to 50 nm, theheight of a wire to 150 nm, and the length of a wire to infinite. In (C)of FIG. 10, the curve “A” is data when the value of (width/pitch) is0.50, and the curve “B” is data when the value of (width/pitch) is 0.33.

From (A) of FIG. 10, it is determined that the pitch of a wire isdesirably equal to or less than 200 nm, the height of a wire isdesirably equal to or greater than

-   5×10⁻⁸ m (50 nm),

and the value of (width/pitch) of a wire is desirably equal to orgreater than 0.33, in order to set the extinction ratio to 10 orgreater. Furthermore, the number of wires is preferably 10 or more.

In addition, the relationship between the length of two wires,

-   the wavelength (λ)

of incident light, and an extinction ratio is shown in FIG. 11.Furthermore, the width of the wires is set to 50 nm, the height of thewires to 150 nm, and the pitch of the wires to three times the width ofthe wires. In FIG. 11, “A” is data in the case of the length of

-   1 μm,

“B” is data in the case of the length of

-   2 μm,

“C” is data in the case of the length of

-   3 μm,

“D” is data in the case of the length of

-   4 μm,

“E” is data in the case of the length of

-   5 μm,-   “F” is data in the case of the length of-   6 μm,-   and “G” is data in the case of the length of being infinite. From    FIG. 11, it is determined that the length of the wires is desirably    equal to or longer than-   2 μm,-   preferably equal to or longer than-   3 μm,-   in order to set the extinction ratio to 10 or greater. Moreover, for    the reason of ease in processing, it is determined that the material    composing a wire is desirably aluminum or an aluminum alloy.

Embodiment 6

Embodiment 6 is also a modification of Embodiment 1. A conceptualdiagram of an imaging element array with a Bayer arrangement in animaging device of Embodiment 6 is shown in FIG. 12, but a first pixelgroup is constituted by one unit pixel row, and one third area 151 andone fourth area 152 are disposed in one pixel. To be more specific, thethird area 151 is disposed in one of two green imaging elements G thatsense green, the fourth area 152 is disposed in the other within onepixel, and moreover, one third area 151 and one fourth area 152 aredisposed in one pixel in the first pixel group constituted by one unitpixel row selected for every N-th row (where N=2^(n), and in the exampleshown in the diagram, n=2).

Alternatively, a conceptual diagram of an imaging element array with aBayer arrangement in a modified example of the imaging device ofEmbodiment 6 is shown in FIG. 13, but one third area 151 and one fourtharea 152 are disposed in one pixel along the first direction.Furthermore, N=8. In addition, a first pixel group is constituted by twounit pixel rows. However, the third area 151 is disposed in one unitpixel row for every second pixel, and the fourth area 152 is disposed inthe other unit pixel row for every second pixel. In one unit pixel row,an imaging element where the third area 151 is not disposed is includedin the second imaging element group, and in the other unit pixel row, animaging element where the fourth area 152 is not disposed is included inthe second imaging element group.

Except for the above points, as an imaging method using the imagingdevice of Embodiment 6 can be the same as that described in Embodiment1, detailed description thereof will be omitted.

Hereinabove, the present invention is described based on preferableembodiments, but the invention is not limited to those embodiments. Theconfigurations and structures of the imaging device and imaging elementsdescribed in the embodiments are examples, and can be appropriatelymodified. For example, as a schematic partial cross-sectional diagram isshown in (A) of FIG. 14, the imaging element 43A can be configured to becomposed of the photoelectric conversion element 61 provided on thesilicon semiconductor substrate 60, and the first flattening film 62,the inorganic insulating base layer 66, the wire-grid polarizer 67, thesecond flattening film 65, the color filter 63, and the on-chip lens 64stacked on the photoelectric conversion element 61. Alternatively, as aschematic partial cross-sectional diagram is shown in (B) of FIG. 14,the imaging element 43A can be configured to be composed of thephotoelectric conversion element 61 provided on the siliconsemiconductor substrate 60, and the first flattening film 62, theon-chip lens 64, the second flattening film 65, the color filter 63, theinorganic insulating base layer 66, and the wire-grid polarizer 67stacked on the photoelectric conversion element 61. The imaging element43B can have the same configuration and structure as those of theimaging element 43A except that the wire-grid polarizer 67 is notprovided. In addition, the imaging element may be the surfaceirradiation type as shown in the drawings, and may be the rear surfaceirradiation type although not shown in the drawings.

A stereoscopic image is displayed based on the right-eye image data andthe left-eye image data obtained by the imaging method of the invention,but as such a display method, a method in which left and right-eyeimages are displayed respectively by installing a circularly-polarizingor linearly-polarizing filter in two projectors, and images are viewedwith circularly-polarized or linearly-polarized glasses corresponding tothe display, a lenticular lens method, and a parallax barrier method canbe exemplified. Furthermore, if images are viewed without usingcircularly-polarized or linearly-polarized glasses, generaltwo-dimensional (flat) images can be viewed. In addition, the processprocedure described above may be understood as a method with such aseries of procedures, and may be understood as a program to cause acomputer to execute the series of procedures or a recording mediumstoring the program. As a recording medium, for example, CDs (CompactDiscs), MDs (MiniDiscs), DVDs (Digital Versatile Discs), memory cards,Blu-ray Discs (registered trademark), or the like can be used.

REFERENCE SIGNS LIST

PG₁ . . . First pixel group, PG₂ . . . Second pixel group, 110 . . .Imaging device, 11 . . . Camera main body, 12 . . . Image processingmeans, 13 . . . Image storage unit, 20 . . . Lens system, 21 . . .Capturing lens, 22 . . . Diaphragm, 23 . . . Imaging lens, 130, 230, and330 . . . First polarization means, 131, 231, and 331 . . . First area,132, 232, and 332 . . . Second area, 333 . . . Center area, 40 . . .Imaging element array, 41 . . . First imaging element group, 42 . . .Second imaging element group, 43A and 43B . . . Imaging element, 150 and250 . . . Second polarization means, 151 and 251 . . . Third area, 152and 252 . . . Fourth area, 60 . . . Silicon semiconductor substrate, 61. . . Photoelectric conversion element, 62 . . . First flattening film,63 . . . Color filter, 64 . . . On-chip lens, 65 . . . Second flatteningfilm, 66 . . . Inorganic insulating base layer, 67, 67A, and 67B . . .Wire-grid polarizer, 68, 68A, and 68B . . . Wire

The invention claimed is:
 1. A parallax imaging method comprising:receiving, with a first pixel grow of a pixel matrix, a first polarizedlight of a subject output from a first polarizer and an imaging lens;outputting, with the first pixel group, first electrical signals basedon the first polarized light of the subject output from the firstpolarizer and the imaging lens; further polarizing, with a secondpolarizer, the first polarized light of the subject output from thefirst polarizer and the imaging lens, the second polarizer including afirst polarization area and a second polarization area different fromthe first polarization area, the first polarization area is configuredto further polarize the first polarized light in a first direction, thesecond polarization area is configured to further polarize the firstpolarized light in a second direction different from the firstdirection; receiving, with a second pixel group of the pixel matrix, thefirst polarized light that is further polarized in the first directionby the first polarization area and the first polarized light that isfurther polarized in the second direction by the second polarizationarea; outputting, with the second pixel group, second electrical signalsbased on the polarized light that is further polarized in the firstdirection and the first polarized light that is further polarized in thesecond direction; receiving, with an imaging processor, the firstelectrical signals and the second electrical signals; generating, withthe imaging processor, parallax information based on the firstelectrical signals; generating, with the imaging processor, imageinformation based on the second electrical signals; and generating, withthe imaging processor, one or more stereoscopic images based at least inpart on the parallax information and the image information.
 2. Theparallax imaging method of claim 1, wherein the second pixel group ofthe pixel matrix includes at least a pixel row, and the first pixelgroup of the pixel matrix includes pixel rows that are not included inthe second pixel group.
 3. The parallax imaging method of claim 1,wherein the second pixel group of the pixel matrix includes at least apixel row for every N-th row, where N≧2, the first pixel group of thepixel matrix comprises pixel rows not equal to every N-th row.
 4. Theparallax imaging method of claim 3, wherein an upper limit of N is N=2n,where n is a natural number from 1 to
 5. 5. The parallax imaging methodof claim 4, wherein n=3.
 6. The parallax imaging method of claim 1,wherein the first direction is orthogonal to the second direction. 7.The parallax imaging method of claim 1, wherein the first polarizedlight of the subject output from the first polarizer and the imaginglens has been polarized by the first polarizer having a first area witha first polarization state and a second area with a second polarizationstate that is different than the first polarization state, wherein thefirst area has a first barycenter and the second area has a secondbarycenter, and wherein a distance between the first barycenter and thesecond barycenter is a baseline length of parallax of both eyes.
 8. Aparallax imaging apparatus comprising: a pixel matrix including a firstpixel group and a second pixel group different from the first pixelgroup, wherein the first pixel group is configured to receive a firstpolarized light of a subject output from a first polarizer and animaging lens, and output first electrical signals based on the firstpolarized light of the subject output from the first polarizer and theimaging lens; a second polarizer including a first polarization area anda second polarization area different from the first polarization area,the first polarization area is configured to further polarize the firstpolarized light of the subject output from the first polarizer and theimaging lens in a first direction, the second polarization area isconfigured to further polarize the first polarized light of the subjectoutput from the first polarizer and the imaging lens in a seconddirection different from the first direction, wherein the second pixelgroup is configured to receive the first polarized light that is furtherpolarized in the first direction by the first polarization area and thefirst polarized light that is further polarized in the second directionby the second polarization area, and output second electrical signalsbased on the first polarized light that is further polarized in thefirst direction and the first polarized light that is further polarizedin the second direction; and an imaging processor configured to receivethe first electrical signals and the second electrical signals, generateparallax information based on the first electrical signals, generateimage information based on the second electrical signals, and generateone or more stereoscopic images based at least in part on the parallaxinformation and the image information.
 9. The parallax imaging apparatusof claim 8, wherein the parallax imaging apparatus is one of a digitalcamera, a personal computer, a mobile terminal equipment, a videocamera, or a game machine.
 10. The parallax imaging apparatus of claim8, wherein the second pixel group of the pixel matrix includes at leasta pixel row, and the second first pixel group of the pixel matrixincludes pixel rows that are not included in the first second pixelgroup.
 11. The parallax imaging apparatus of claim 8, wherein the firstsecond pixel group includes of at least a pixel row for every N-th row,where N≧2, and the second first pixel group comprises pixel rows notequal to every N-th row.
 12. The parallax imaging apparatus of claim 8,wherein the second pixel group of the pixel matrix includes at least apixel row, and the first pixel group of the pixel matrix includes pixelrows that are not included in the second pixel group.
 13. The parallaximaging apparatus of claim 8, wherein the second pixel group of thepixel matrix includes at least a pixel row for every N-th row, whereN≧2, the first pixel group of the pixel matrix comprises pixel rows notequal to every N-th row.
 14. The parallax imaging apparatus of claim 13,wherein an upper limit of N is N=2n, where n is a natural number from 1to
 5. 15. The parallax imaging apparatus of claim 14, wherein n=3. 16.The parallax imaging apparatus of claim 8, wherein the second directionis orthogonal to the first direction.
 17. The parallax imaging apparatusof claim 8, wherein the first polarized light of the subject output fromthe first polarizer and the imaging lens has been polarized by the firstpolarizer having a first area with a first polarization state and asecond area with a second polarization state that is different than thefirst polarization state, wherein the first area has a first barycenterand the second area has a second barycenter, and wherein a distancebetween the first barycenter and the second barycenter is a baselinelength of parallax of both eyes.
 18. A parallax imaging systemcomprising: a pixel matrix including a first pixel group and a secondpixel group different from the first pixel group, the first pixel groupis configured to receive a first polarized light of a subject outputfrom a first polarizer and an imaging lens, and output first electricalsignals based on the first polarized light of the subject output fromthe first polarizer and the imaging lens; and a second polarizerincluding a first polarization area and a second polarization areadifferent from the first polarization area, the first polarization areais configured to further polarize the first polarized light of thesubject output from the first polarizer and the imaging lens in a firstdirection, the second polarization area is configured to furtherpolarize the first polarized light of the subject output from the firstpolarizer and the imaging lens in a second direction different from thefirst direction; wherein the second pixel group is configured to outputsecond electrical signals based on the first polarized light that isfurther polarized in the first direction by the first polarization areaand the first polarized light that is further polarized in the seconddirection by the second polarization area, an imaging processorconfigured to receive the first electrical signals and the secondelectrical signals, generate parallax information based on the firstelectrical signals, generate image information based on the secondelectrical signals, and generate one or more stereoscopic images basedat least in part on the parallax information and the image information.19. The parallax imaging system of claim 18, wherein the second pixelgroup of the pixel matrix includes at least a pixel row, and the firstpixel group of the pixel matrix includes pixel rows that are notincluded in the second pixel group.
 20. The parallax imaging system ofclaim 18, wherein the second pixel group of the pixel matrix includes atleast a pixel row for every N-th row, where N≧2, the first pixel groupof the pixel matrix comprises pixel rows not equal to every N-th row.21. The parallax imaging system of claim 18, wherein wherein the firstpolarized light of the subject output from the first polarizer and theimaging lens has been polarized by the first polarizer having a firstarea with a first polarization state and a second area with a secondpolarization state that is different than the first polarization state,wherein the first area has a first barycenter and the second area has asecond barycenter, and wherein a distance between the first barycenterand the second barycenter is a baseline length of parallax of both eyes.22. A parallax imaging apparatus comprising: a plurality of pixelsincluding a first pixel group and a second pixel group different fromthe first pixel group, the first pixel group is configured to receive afirst polarized light of a subject output from a first polarizer and animaging lens, and output first electrical signals based on the firstpolarized light of the subject output from the first polarizer and theimaging lens; and a second polarizer including a first polarization areaand a second polarization area different from the first polarizationarea, the first polarization area is configured to further polarize apolarized light the first polarized light of a subject the subjectoutput from the first polarizer and the imaging lens in a firstdirection, the second polarization-area is configured to furtherpolarize the first polarized light of the subject output from the—firstpolarizer and the imaging lens in a second direction different from thefirst direction; wherein the second pixel group is configured to receivethe first polarized light that is further polarized in the firstdirection from the second polarizer, receive the first polarized lightthat is further polarized in the second direction from the secondpolarizer, and output second electrical signals based on the firstpolarized light that is further polarized in the first direction and thefirst polarized light that is further polarized in the second direction.23. The parallax imaging apparatus of claim 22, wherein the firstpolarized light of the subject output from the first polarizer and theimaging lens has been polarized by the first polarizer having a firstarea with a first polarization state and a second area with a secondpolarization state that is different than the first polarization state,wherein the first area has a first barycenter and the second area has asecond barycenter, and wherein a distance between the first barycenterand the second barycenter is a baseline length of parallax of both eyes.24. A non-transitory computer readable medium storing a program code fora parallax imaging process, the program code being executable by animaging processor to perform operations comprising: receiving firstelectrical signals from a first pixel group of a pixel matrix, the firstelectrical signals are based on a first polarized light of a subjectoutput from a first polarizer and an imaging lens; receiving firstsecond electrical signals from a first second pixel group of the pixelmatrix, the first second electrical signals are based on a furtherpolarization of a polarized the first polarized light of a subject thesubject output from an imaging the first polarizer and the imaging lensin a first direction and a second further polarization the firstpolarized light of the subject output from the first polarizer and theimaging lens in a second direction different from the first direction;generating parallax information based on the first electrical signals;generating image information based on the second electrical signals; andgenerating one or more stereoscopic images based at least in part on theparallax information and the image information.
 25. The non-transitorycomputer readable medium of claim 24, wherein the first polarized lightof the subject output from the first polarizer and the imaging lens hasbeen polarized by the first polarizer having a first area with a firstpolarization state and a second area with a second polarization statethat is different than the first polarization state, wherein the firstarea has a first barycenter and the second area has a second barycenter,and wherein a distance between the first barycenter and the secondbarycenter is a base-line length of parallax of both eyes.